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The interactive game theoretical approach to the description of perception processes 
is proposed. The subject is treated formally in terms of a new class of the verbal- 
izable interactive games which are called the perception games. An application of 
the previously elaborated formalism of dialogues and verbalizable interactive games 
to the visual perception allows to combine the linguistic (such as formal grammars), 
psycholinguistic and (interactive) game theoretical methods for analysis of the image 
understanding by a human that may be also useful for the elaboration of computer vi- 
sion systems. By the way the interactive game theoretical aspects of interpretational 
\ geometries are clarified. 
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The mathematical formalism of interactive games, which extends one of ordinary 
games [1] and is based on the concept of an interactive control, was recently pro- 
posed by the author [2] to take into account the complex composition of controls 
of a real human person, which are often complicated couplings of his/her cognitive 
and known controls with the unknown subconscious behavioral reactions. This for- 
malism is applicable also to the description of external unknown influences and, 
thus, is useful for problems in computer science (e.g. the semi-artificially controlled 
distribution of resources) and mathematical economics (e.g. the financial games 
with unknown dynamical factors). 

However, the original impetus for the investigations lay in the sphere of human 
visual perception [3]. When the first steps were made it became clear that it 
is important to understand this sphere formally including it into the framework of 
the elaborating interactive game theory. The interactive game theoretical definition 
of dialogues as psycholinguistic phenomena and the description of the verbalizable 
interactive games [4] were crucial to make it possible. This article is an attempt to 
solve the prescribed problem. 
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I. Interactive games and their verbalization 
1.1. Interactive systems and intention fields. 

Definition 1 [2]. An interactive system (with n interactive controls) is a control 
system with n independent controls coupled with unknown or incompletely known 
feedbacks (the feedbacks as well as their couplings with controls are of a so com- 
plicated nature that their can not be described completely). An interactive game 
is a game with interactive controls of each player. 

Below we shall consider only deterministic and differential interactive systems. 
In this case the general interactive system may be written in the form: 

(1) ip = $(^,U 1 ,U 2 , • • • ,«„), 

where ip characterizes the state of the system and Ui are the interactive controls: 

Ui (t) = Ui(u°(t), [¥>(r)]| T<t ), 

i.e. the independent controls u°(t) coupled with the feedbacks on [<^(t)]| t ^. One 
may suppose that the feedbacks are integrodifferential on t. 

Proposition [2]. Each interactive system (1) may be transformed to the form (2) 
below (which is not, however, unique): 

(2) <p = *(<p,Z), 

where the magnitude £ (with infinite degrees of freedom as a rule) obeys the equation 

(3) £ = H(f,p,ui,u 2 ,... ,«„), 

where Ui are the interactive controls of the form Ui(t) = Ui(u°(t); ip(t),£(t)) (here 
the dependence of ui on and tp{t) is differential on t, i.e. the feedbacks are 
precisely of the form u^t) = <p(t),£(t), <p(t), £(£), . . . ,p {k) (t), 

e k \t)))- 

Remark 1. One may exclude cp(t) from the feedbacks in the interactive controls 
Ui(t). One may also exclude the derivatives of £ and tp on t from the feedbacks. 

Definition 2 [2]. The magnitude £ with its dynamical equations (3) and its cont- 
ribution into the interactive controls Ui will be called the intention field. 

Note that the theorem holds true for the interactive games. In practice, the inten- 
tion fields may be often considered as a field-theoretic description of subconscious 
individual and collective behavioral reactions. However, they may be used also the 
accounting of unknown or incompletely known external influences. Therefore, such 
approach is applicable to problems of computer science (e.g. semi-automatically 
controlled resource distribution) or mathematical economics (e.g. financial games 
with unknown factors). The interactive games with the differential dependence of 
feedbacks are called differential. Thus, the theorem states a possibility of a reduc- 
tion of any interactive game to a differential interactive game by introduction of 
additional parameters - the intention Gelds. 
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1.2. Some generalizations. The interactive games introduced above may be 
generalized in the following ways. 

The first way, which leads to the indeterminate interactive games, is based on 
the idea that the pure controls u°(t) and the interactive controls Ui(t) should not be 
obligatory related in the considered way. More generally one should only postulate 
that there are some time-independent quantities F a (ui(t),u°(t),ip(t), . . . ,(p( k \t)) 
for the independent magnitudes Ui(t) and u°(t). Such claim is evidently weaker 
than one of Def.l. For instance, one may consider the inverse dependence of the 
pure and interactive controls: u°(t) = u° (ui(t) , (p(t) , . . . ,(p( k \t)). 

The second way, which leads to the coalition interactive games, is based on 
the idea to consider the games with coalitions of actions and to claim that the 
interactive controls belong to such coalitions. In this case the evolution equations 
have the form 

<p = ui, . . . ,v m ), 

where Vi is the interactive control of the z-th coalition. If the i-th coalition is defined 
by the subset Ii of all players then 

Vi = Vi((p(t),... ,<p (k) (t),u?\j G Ii). 

Certainly, the intersections of different sets Ii may be non-empty so that any player 
may belong to several coalitions of actions. Def.l gives the particular case when 
Ii = {i}- 

The coalition interactive games may be an effective tool for an analysis of the 
collective decision making in the real coalition games that spread the applicability 
of the elaborating interactive game theory to the diverse problems of sociology. 

1.3. Differential interactive games and their e— representations. 

Definition 3 [4]. The e -representation of differential interactive game is a repre- 
sentation of the differential feedbacks in the form 

(4) Ui(t) = Ui(u°i,ip(t),... ,^ k \t);ei(t)) 

with the known function Ui of all its arguments, where the magnitudes £i(t) G £ 
are unknown functions of u° and ip(t) with its higher derivatives: 

e i (t) = e i (u° i (t),<p(t),<p(t),... ,f {k) (t)). 

It is interesting to consider several different ^-representations simultaneously. 
For such simultaneous ^-representations with ^-parameters e\ a ^ a crucial role is 
played by the time-independent relations between them: 

p _(«) SN) o (fc)\ — ft 

t P\ e i ,Wj,VJ, . . . ,(/7 w J = U, 

which are called the correlation integrals. Certainly, in practice the correlation inte- 
grals are determined a posteriori and, thus they contain an important information 
on the interactive game. Using the sufficient number of correlation integrals one is 
able to construct various algebraic structures in analogy to the correlation functions 
in statistical physics and quantum field theory. 
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1.4. Dialogues as interactive games. The verbalization. 

Dialogues as psycholinguistic phenomena can be formalized in terms of inter- 
active games. First of all, note that one is able to consider interactive games of 
discrete time as well as interactive games of continuous time above. 

Defintion 4A (the naive definition of dialogues) [4]. The dialogue is a 2- 
person interactive game of discrete time with intention fields of continuous time. 

The states and the controls of a dialogue correspond to the speech whereas the 
intention fields describe the understanding. 

Let us give the formal mathematical definition of dialogues now. 

Definition 4B (the formal definition of dialogues) [4]. The dialogue is a 
2-person interactive game of discrete time of the form 

(5) (fin = $(¥>n-l,tfn,f(T)|*n-l<T^*n)- 

Here ip n = <p(t n ) are the states of the system at the moments t n (t < t\ < t 2 < 
. . . <t n < . . . ), v n = v(t n ) = (vi(t n ), V2(t n )) are the interactive controls at the same 
moments; £(r) are the intention fields of continuous time with evolution equations 

(6) i(t) = E(t(t),m), 

where u(t) = (u\(t) , U2(t)) are continuous interactive controls with ^-represented 
couplings of feedbacks: 

u i {t) = u i {u°{t),Z{t);e i (t)). 

The states Lp n and the interactive controls v n are certain known functions of the 
form 

Cp n =(p n (£(T)^(r)\t n - 1 ^T^t n ), 

v n =t7 n (tr(r),^(r)|t n _i^r^t n ). 

Note that the most nontrivial part of mathematical formalization of dialogues 
is the claim that the states of the dialogue (which describe a speech) are certain 
"mean values" of the e-parameters of the intention fields (which describe the un- 
derstanding) . 

Important. The definition of dialogue may be generalized on arbitrary number of 
players and below we shall consider any number n of them, e.g. n = 1 or n = 3, 
though it slightly contradicts to the common meaning of the word "dialogue" . 

An embedding of dialogues into the interactive game theoretical picture generates 
the reciprocal problem: how to interpret an arbitrary differential interactive game 
as a dialogue. Such interpretation will be called the verbalization. 

Definition 5 [4] . A differential interactive game of the form 

with e-represented couplings of feedbacks 
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is called verbalizable if there exist a posteriori partition t < t\ < t 2 < . . . < t n < . . . 
and the integro differential functionals 

w n (e(r),v?(r)|t n _i^r^t n ), 
tT n (M°(r),^(r)|t n _i^r^i n ) 

such that 

(9) u n = ft(w n _i,v n ;¥>(r)|* n _i^T<t n ). 

The verbalizable differential interactive games realize a dialogue in sense of Def.4. 

The main heuristic hypothesis is that all differential interactive games "which 
appear in practice" are verbalizable. The verbalization means that the states of a 
differential interactive game are interpreted as intention fields of a hidden dialogue 
and the problem is to describe such dialogue completely. If a differential interactive 
game is verbalizable one is able to consider many linguistic (e.g. the formal grammar 
of a related hidden dialogue) or psycholinguistic (e.g. the dynamical correlation of 
various implications) aspects of it. 

During the verbalization it is a problem to determine the moments ti. A way 
to the solution lies in the structure of e-representation. Let the space E of all 
admissible values of e-parameters be a CW-complex. Then ti are just the moments 
of transition of the e-parameters to a new cell. 

II. Perception games and the image understanding 

Let us considered a verbalizable interactive game. We shall suppose for simplicity 
that the concrete set is finished if some quantity F(u n , tp(t)) reaches some critical 
value Fq. The game will be called perception game iff the moments ti are just the 
moments of finishing of the concrete sets so the multistage perception game realizes 
a sequence of sets with initial states coinciding with the final state of the preceeding 
set. Such construction is not senseless contrary to the most of the ordinary games 
because the quantity F should be recalculated with the new u. Thus, we have the 
following general definition. 

Definition 6. The perception game is a multistage verbalizable game (no matter 
finite or infinite) for which the intervals [£j,£j+i] are just the sets. The conditions 
of their finishing depends only on the current value of (p and the state of u at 
the beginning of the set. The initial position of the set is the final position of the 
preceeding one. 

Practically, the definition describes the discrete character of the perception and 
the image understanding. For example, the goal of a concrete set may be to perceive 
or to understand certain detail of the whole image. Another example is a continuous 
perception of the moving or changing object. 

Note that the definition of perception games is applicable to various forms of 
perception. However, the most interesting one is the visual perception. Besides the 
numerous problems of human visual perception of reality (as well as of computer 
vision) there exists a scope of numerous questions of the human behaviour in the 
computer modelled worlds, e.g. constructed by use of the so-called "virtual reality" 
(VR) technology. There are no an evident boundary between them because we 
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can always interpret the internal space of our representations as a some sort of 
the natural "virtual reality" and apply the analysis of perception in VR to the 
real image understanding as well as to the activity of imagination. So one should 
convince that it is impossible to explain all phenomena of our visual perception of 
reality without deep analysis of its peculiarities in the computer modelled worlds 
(cf.[5]). Especially crucial role is played by the so-called integrated realities (IR), 
in which the channels only of some kinds of perception are virtual (e.g. visual) 
whereas others are real (e.g. tactile, kinesthetic). 

The proposed definition allows to take into account the dialogical character of the 
image understanding and to consider the visual perception, image understanding 
and the verbal (and nonverbal) dialogues together. It may be extremely useful for 
the analysis of collective perception, understanding and controlling processes in the 
dynamical environments - sports, dancings, martial arts, the collective controlling 
of moving objects, etc. 

On the other hand this definition explicates the self-organizing features of human 
perception, which may be unraveled by the game theoretical analysis. 

And, finally, the definition put a basis for a systematical application of the 
linguistic (e.g. formal grammars) and psycholinguistic methods to the image un- 
derstanding as a verbalizable interactive game with a mathematical rigor. 

Also interpreting perception processes and the image understanding as the ver- 
balizable interactive games we obtain an opportunity to adapt some procedures of 
the image understanding to the verbalizable interactive games of a different nature, 
e.g. to the verbal dialogues. It may enlight the processes of generation of subjective 
figurative representations, which is important for the analysis of the understanding 
of speech in dialogues. 

Traditionally the problems of the visual perception are related to geometry (as 
descriptive as abstract) so it is reasonable to pay an attention to the geometrical 
background for the perception videogames and then to combine both geometrical 
and interactive game theoretical approaches to the visual perception and the image 
understanding. 

III. Interpretational geometries, intentional anomalous virtual 

REALITIES AND THEIR INTERACTIVE GAME THEORETICAL ASPECTS 

3.1. Interpretational figures [6]. Geometry described below is related to a 
class of interactive information systems. Let us call an interactive information sys- 
tem computer graphic (or interactive information videosystem) if the information 
stream "computer-user" is organized as a stream of geometric graphical data on a 
screen of monitor; an interactive information system will be called psychoinforma- 
tion if an information transmitted by the channel "user-computer" is (completely 
or partially) subconscious. In general, an investigation of interactively controlled 
(psychoinformation) systems for an experimental and a theoretical explication of 
possibilities contained in them, which are interesting for mathematical sciences 
themselves, and of "hidden" abstract mathematical objects, whose observation and 
analysis are actually and potentially realizable by these possibilities, is an impor- 
tant problem itself. So below there will be defined the notions of an interpretational 
figure and its symbolic drawing that undoubtly play a key role in the description 
of a computer-geometric representation of mathematical data in interactive in- 
formation systems. Below, however, the accents will be focused a bit more on 
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applications to informatics preserving a general experimentally mathematical view, 
the interpretational figures (see below) will be used as pointers to droems and in- 
teractive real-time psychoinformation videosystems will be regarded as components 
of integrated interactive videocognitive systems for accelerated nonverbal cognitive 
communications. 

In interactive information systems mathematical data exist in the form of an 
interrelation between the geometric internal image (figure) in the subjective space of 
the observer and the computer-graphic external representation. The latter includes 
visible (drawings of the figure) and invisible (analytic expressions and algorithms 
for constructing these images) elements. Identifying geometric images (figures) in 
the internal space of the observer with computer-graphic representations (visible 
and invisible elements) is called a translation, in this way the visible object may be 
not identical with the figure, so that separate visible elements may be considered 
as modules whose translation is realized independently. The translation is called 
an interpretation if the translation of separate modules is performed depending on 
the results of the translation of preceding ones. 

Definition 7. The figure obtained as a result of interpretation is called an inter- 
pretational figure. 

Note that the interpretational figure may have no usual formal definition; namely, 
only if the process of interpretation admits an equivalent process of compilation 
definition of the figure is reduced to definitions of its drawings that is not true 
in general. So the drawing of an interpretational figure defines only dynamical 
"technology of visual perception" but not its "image" , such drawings will be called 
symbolic. 

The computer-geometric description of mathematical data in interactive infor- 
mation systems is closely connected with the concept of anomalous virtual reality. 

3.2. Intentional anomalous virtual realities [3,6]. 

Definition 8. (A). Anomalous virtual reality {AVE) in a narrow sense means 
some system of rules of a nonstandard descriptive geometry adapted for realiza- 
tion on video computers (or multisensorial systems of "virtual reality"). Anomalous 
virtual reality in a wide sense also involves an image in cyberspace formed in ac- 
cordance with said system of rules. We shall use the term in its narrow sense. (B). 
Naturalization is the constructing of an AVR from some abstract geometry or phys- 
ical model. We say that anomalous virtual reality naturalizes the abstract model 
and the model trans cendizes the naturalizing anomalous virtual reality. (C). Visu- 
alization is the constructing of certain image or visual dynamics in some anomalous 
virtual reality (realized by hardware and software of a computer-grafic interface of 
the concrete videosystem) from the objects of an abstract geometry or processes 
in a physical model. (D). Anomalous virtual reality, whose objects depend on the 
observer, is called an intentional anomalous virtual reality {IAVE). The general- 
ized perspective laws for IAVR contain the interactive dynamical equations for the 
observed objects in addition to standard (geometric) perspective laws. In IAVR 
the observation process consists of a physical process of observation and a virtual 
process of intentional governing of the evolution of images in accordance with the 
dynamical perspective laws. 

In intentional anomalous virtual reality (IAVR) that is realized by hardware and 
software of the computer- graphic interface of the interactive videosystem being geo- 
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metrically modelled by this IAVR (on the level of descriptive geometry whereas the 
model transcendizing this IAVR realizes the same on the level of abstract geometry) 
respectively, the observed objects are demonstrated as connected with the observer 
who acts on them and determines, or fixes, their observed states so that the objects 
are thought only as a potentiality of states from the given spectrum whose real- 
ization depends also on the observer. The symbolic drawings of interpretational 
figures may be considered as states of some IAVR. 

Note that mathematical theory of anomalous virtual realities (AVR) including 
the basic procedures of naturalization and thanscending connected AVR with the 
abstract geometry is a specific branch of modern nonclassical descriptive (computer) 
geometry. 

Definition 8E. The set of all continuously distributed visual charcteristics of the 
image in anomalous virtual reality is called an anomalous color space; the anoma- 
lous color space elements of noncolor nature are called overcolors, and the quan- 
tities transcendizing them in an abstract model are called "latent lights". The set 
of the generalized perspective laws in a fixed anomalous color space is called a 
color-perspective system. 

3.3. Remarks on the interactive game theoretical aspects. Certainly, the 
interpretational geometries may be considered as the perception games. An inter- 
esting geometrical consequence of such approach was proposed [7]. 

Proposition. There exist models of interpretational geometries in which there are 
interpretational figures observed only in a multi-user mode. 

It seems that this proposition may be regarded as a startpoint for the future 
interactions between geometry and interactive game theory in the sphere of math- 
ematical foundations for the collective perception and image understanding of real 
objects as well as objects in the computer VR or IR systems. 

IV. Conclusions 

Thus, the interactive game theoretical approach to the description of perception 
processes is proposed. A new class of the multistage verbalizable interactive games, 
the perception games, is introduced. The interactive game theoretical aspects of 
interpretational geometries are clarified. Perspectives are sketched. 
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